Systems for localization of targets inside a body

ABSTRACT

The present disclosure relates, in part, to a scanning sufficiency apparatus that computes whether a handheld scanning device has scanned a volume for a sufficiently long time for there to be detections and then indicate to the user that the time is sufficient in 3-D rendered voxels. Also described is a hand held medical navigation apparatus with system and methods to map targets inside a patient&#39;s body.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/080,184, filed Nov. 14, 2014, the teachings of which are herebyincorporated by reference in its entirety for all purposes.

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSOREDRESEARCH AND DEVELOPMENT

Not Applicable

BACKGROUND

Generally, embodiments of the present application relate to apparatusesused in image guided surgery techniques. Specifically, this applicationrelates to medical imaging instruments and the use of data fromdetectors taken pre- and intra-operatively to guide medical procedures.Such procedures include tissue excision, tissue biopsy, injections,simulations or implantations.

Image guided surgery apparatuses are used to assist surgeons performingmedical procedures. These apparatuses generally use tracked surgicalinstruments in conjunction with pre-operative and intra-operative imagedatasets with the purpose of representing the position and orientationof the surgical instruments in respect to the adjacent anatomy.

Some existing tracking instruments use overhead optical stereoscopicsystems in conjunction with infrared reflective spheres attached toinstruments and patient. Other devices use camera systems attached tothe medical imaging systems to track the position and orientation ofthese medical instruments with respect to other medical instruments,such as needles. The function of these devices often depend on one ormore instruments remaining in the line-of-sight of the tracking camerato determine the correct position and orientation of the medicalinstrument. In the course of a standard procedure, the movement of thesurgeon, assisting medical personnel, or of medical instruments andequipment can obstruct the line of sight of the tracking camera,resulting in poor tracking and imaging data. Furthermore, overheadoptical tracking systems require more equipment inside the room, whichcan hinder the procedure being performed.

As such, there is a need in the art for more accurate and precisemedical navigation devices.

BRIEF SUMMARY

Generally, the embodiments of the present invention relate to a scanningsufficiency device that is able to instantaneously report to a user howwell a particular area of a subject's body has been scanned. The devicesdescribed herein improve scanning quality and reduce errors by betterdefining the precise location of a detected signal.

Embodiments can also relate to hand held medical navigation apparatuseswith systems and methods to map targets inside the body. The apparatusesdescribed herein use tracking systems, such as cameras, attached tomedical sensing instruments to track the position of the medical sensinginstruments with respect to the patient. The apparatus sometimes has atracking system with a lateral view.

The images captured by the tracking camera or scanning data from amedical sensing instrument can be presented to the operator on a screen,on a head mounted display, or using other visualization device such asprojectors. Data sets collected by the tracking camera and the medicalsensing instrument can also be combined to create an overlaid model thatcan be presented on the above mentioned visualization means. Variousaspects of the information collected by the apparatuses can be presentedin a graphical user interface that can improve the quality of a scan.

In some embodiments, the present invention relates to a scanningsufficiency apparatus. The scanning sufficiency apparatus includes, anionizing radiation sensor within a housing assembly, a tracking systemproviding the position and orientation of the sensor with respect to anexamined object, a visualization device operably linked to the at leastone processor to show an instantaneous image, at least one processor anda memory operatively coupled with the sensor and the tracking system,the memory having instructions for execution by the at least oneprocessor configured to associate scanning data from the sensor with thecalculated spatial position and orientation of the sensor with respectto the object to create registered scans, separate an adjacentvolumetric space into 3-D imaging elements, produce a 3-D model ofradioactive sources by combining the registered scans, calculate ascanning completeness of a particular 3-D imaging element by calculatinga scanning completeness value (SCV), create a map of SCV values, andcreate the instantaneous image comprising a rendering of the SCV map,wherein the calculation of the SCV takes into consideration the sensortracking information to inform the user about the partial volumes thathave been scanned enough and the partial volumes that have not beenscanned enough for a pre-defined scanning objective

In some embodiments, the present invention also includes an imageformation and navigation apparatus. The image formation and navigationapparatus includes, a housing assembly, an ionizing radiation sensor atleast partially enclosed within the housing assembly and disposedtowards the distal end of the housing assembly, an optical camera atleast partially enclosed within the housing assembly, the trackingcamera having a field of view that overlaps partially with the field ofview of the sensor; a visualization device operably linked to at leastone processor and configured to show instantaneous renderings of images,the at least one processor operatively coupled with a memory, the sensorand the camera, the memory having instructions for execution by the atleast one processor configured to determine a pose of the camera withrespect to an object using an image captured by the camera and, usingtransformations derived from the camera being rigidly connected with thesensor, determine a spatial position and orientation of the sensor withrespect to the object, associate scanning data from the sensor with thecalculated spatial position and orientation of the sensor with respectto the object to create registered scans, create a 3-D map of sourcesgenerating signatures measured by the sensor by using at least tworegistered scans, create an image for visualization on the visualizationdevice which is a combination of a rendered image of the 3-D map and animage processed from an image captured by the camera, create and sendfor visualization on the visualization device renderings of 1-D and 2-Dprojections of the 3-D map along or perpendicular to the housingassembly preferred axis.

In some embodiments, the present invention provides a laparoscopic orintra-cavity image formation and navigation apparatus, comprising ahousing assembly having an elongated, essentially cylindrical part witha diameter of less than 30 mm, a gamma ray sensor with spectroscopic andposition resolution at least partially enclosed within the elongatedpart of the housing assembly, towards the distal end of the elongatedpart, a tracking and co-registration system to track the position andorientation of the sensor and the camera with respect to examinedorgans, an optical camera at least partially enclosed within anotherhousing assembly having an elongated, essentially cylindrical part witha diameter of less than 30 mm, towards the distal end of the elongatedpart, the tracking camera having a field of view that observes thegeneral area where the sensor is operated, the at least one processoroperatively coupled with a memory, the sensor, the tracking andco-registration system, and the camera, the memory having instructionsfor execution by the at least one processor configured to associategamma ray interaction data from the sensor with the calculated spatialposition and orientation of the sensor with respect to the object tocreate registered scans, determine a scattering angle around ascattering direction of a gamma ray interacting at least two times inthe sensor system by resolving the kinematics of the gamma rayinteractions within the sensor system, create a 3-D map of gamma raysources by resolving statistically the intersection of at least twospatially registered cones formed by the determined scattering anglearound the scattering direction.

In some embodiments the present invention relates to a radiationposition sensitive apparatus. The radiation position sensitive apparatusincludes an elongated housing assembly having a longitudinal axis, agamma ray probe at least partially enclosed within the elongated housingassembly and disposed along the longitudinal axis of the elongatedhousing assembly, a tracking camera at least partially enclosed withinthe elongated housing assembly, the tracking camera having a trackingfield of view that is lateral to the longitudinal axis, the trackingcamera disposed at a predetermined proximity to the gamma ray probe, atleast one processor and a memory operatively coupled with the trackingcamera, the memory having instructions for execution by the at least oneprocessor for calculating a spatial position and orientation of lateralimages taken by the tracking camera with respect to a fiducial andassociating scanning data from the gamma ray probe with the calculatedspatial position and orientation of the lateral images to determine aposition and orientation of the scanning data using the known proximityof the gamma ray probe to the tracking camera.

The radiation position sensitive apparatus can include a memory that hasinstructions for execution by the at least one processor configured toconvert the scanning data to a reconstructed diagram identifying arelative location of a radiation source to the fiducial. In someembodiments, the reconstructed diagram is produced using Comptonimaging, self-collimation effects, and proximity imaging.

The radiation position sensitive apparatus can include a memory that hasinstructions for execution by the at least one processor configured tocombine the lateral images and the reconstructed diagram to produce athree dimensional (3-D) model of a subject's tissue.

The radiation position sensitive apparatus can include a display screenoperably linked to the at least one processor and configured to showinstantaneous renderings of the 3-D model. The display screen can alsobe configured to show instantaneous renderings of the reconstructeddiagram.

In some embodiments the fiducial marker is a marker with a binary codingapplied to an area of interest.

The radiation position sensitive apparatus can include a transparentoptical window along a side of the elongated housing through which thetracking camera is configured to see the tracking field of view. Theoptical window can also include a device configured for supplying astream of fluid in order to keep the transparent optical window clear.The fluid can include air or liquid.

The radiation position sensitive apparatus can include an illuminationsource with an illumination field of view that overlaps with thetracking field of view. The illumination source can be an optical fiber.

The radiation position sensitive apparatus can include a transparentillumination window wherein the illumination source illuminates thetracking field of view through the transparent illumination window. Theillumination window can also include a device configured for supplying astream of fluid to keep the transparent illumination window clear. Thefluid can be air or liquid.

The illumination source can be spatially patterned light, spectrallycoded light, time coded light, uniform light, and combinations thereof.

The gamma ray probe in the radiation position sensitive apparatus caninclude an enclosed matrix surrounding a sensor. The material of thesensor can be cadmium zinc tellurium (CdZnTe) detector, a positionsensitive scintillator, a segmented silicon (Si) detector, a depletedcharge-coupled device (CCD) sensor, a depleted complementary metal-oxidesemiconductor (CMOS) sensor, or any other known sensor in the art.

The sensor of the gamma ray probe in the radiation position sensitiveapparatus includes a first sensor and can include a second sensorproximate to the first sensor.

In another aspect, the present invention includes a medical navigationapparatus including an elongated housing assembly having a longitudinalaxis, a sensor probe at least partially enclosed within the elongatedhousing assembly and disposed along the longitudinal axis of theelongated housing assembly, a tracking camera at least partiallyenclosed within the elongated housing assembly, the tracking camerapointing outward from the longitudinal axis, the tracking cameradisposed at a predetermined position and orientation with respect to theprobe, at least one processor and a memory operatively coupled with thetracking camera, the memory having instructions for execution by the atleast one processor for calculating a spatial position and orientationof the tracking camera with respect to a fiducial based on an image thatincludes the fiducial, computing a spatial position and orientation ofthe sensor probe, and associating scanning data from the probe with thecomputed spatial position and orientation of the sensor probe.

The sensor prove of the medical navigation apparatus can be a magneticsensor, an electromagnetic sensor, a gamma ray detector, or any otherknown sensor useful in the art.

In another aspect, the present invention includes a scanningcompleteness apparatus including a detector, a tracking camera rigidlyattached to the detector and being a known proximity to the detector, atleast one processor and a memory operatively coupled with the detectorand the tracking camera, the memory having instructions for execution bythe at least one processor configured for calculating a spatial positionand orientation of images taken by the tracking camera with respect to afiducial marker, associating scanning data from the detector with thecalculated spatial position and orientation of the images to determine aposition and orientation of the scanning data using the known proximityof the detector to the tracking camera, producing a three dimensional(3-D) model of a subject's tissue detected by the tracking camera andthe relative location of one or more signals detected by the detector,and separating volumetric units of the 3-D model into voxels, whereinscanning completeness of a particular voxel is determined byinstantaneously calculating a scanning completeness value (SCV).

The scanning completeness value of the scanning completeness apparatuscan be determined by summing the probability that the signal emitted bya marker inside the voxel is detected by the detector at each moment oftime over the scanning period.

The scanning completeness apparatus can also include a display screenoperably linked to the at least one processor to show an instantaneousimage of the 3-D model. The 3-D model can be shown as a 3-D mesh of thepatient body. The 3-D model can also include an overlapped view of thescanning data and the images.

In some embodiments, the scanning completeness apparatus includes anotification to the user that the scanning of a particular voxel issufficient. The indication to the user includes an audible sound, acolor change in a voxel, or a conversion of a voxel from opaque totransparent.

In some embodiments the fiducial marker is a marker with a binary codingapplied to an area of interest.

The detector of the scanning sufficiency device can be a radiationdetector, an electromagnetic sensor, a magnetic sensor, an ultrasounddevice, or any other detector useful in the art.

The camera of the scanning sufficiency device can include an opticalcamera. The optical camera can include a visible light camera or aninfrared (IR) camera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a hand held medical navigation apparatus inaccordance with an embodiment.

FIG. 2 illustrates the use of a medical navigation apparatus inaccordance with an embodiment.

FIG. 3 illustrates a cross sectional view of a medical navigationapparatus in accordance with an embodiment.

FIG. 4A illustrates use of a medical navigation apparatus in accordancewith an embodiment.

FIG. 4B illustrates use of a medical navigation apparatus in accordancewith an embodiment.

FIG. 5A illustrates a model showing combined images from the trackingcamera and scanning data from the detector in accordance with anembodiment.

FIG. 5B illustrates a cross section of a model showing combined imagesfrom the tracking camera and scanning data from the detector inaccordance with an embodiment.

FIG. 6 illustrates a view of a graphical user interface in accordancewith an embodiment.

FIG. 7 illustrates the use of a medical scanning apparatus and anexternal tracking camera in accordance with an embodiment.

DETAILED DESCRIPTION

Generally, medical tracking apparatuses are described herein.Particularly described are exemplary apparatuses used to accuratelydetermine the position of targets of interest inside the body in respectto the body of the patient or a fiducial. For example apparatusesdescribed herein can be used to provide information about criticaltissue, such as veins, arteries and nerves, that may be present on thepath towards those targets, so that surgical instruments handled by theoperator will not damage them. Further uses for the apparatusesdescribed herein include identifying the location of a tissue to beremoved, such as a tumor, or to be used post-operatively to determine ifthe excision of a particular tissue was performed successfully.

Targets of interest inside a subject's body can be labeled by any knownsignal in the art. Such signals include, but are not limited to, aTc-99m tracer, a radioactive seed or other radioactive tracer, magneticnano-particles, micro-bubbles, and fludeoxyglucose (FDG). A person ofskill in the art will recognize that different types of detectors arenecessary to detect these exemplary signals.

FIG. 1 illustrates a hand held medical navigation apparatus 100 thatincludes a housing 102, a position sensitive detector 104, a trackingcamera 106, and a handle 111. The tracking camera 106 has a trackingfield of view 105 and is at a predetermined proximity to the positionsensitive detector 104.

The medical navigation apparatus 100 includes at least one processor(not shown) and a memory (not shown), either within the housing 102 orin an external processing unit, that is coupled with the tracking camera106. Using instructions provided within the memory, the processordetermines the spatial position and orientation of images captured bythe tracking camera 106 with respect to one or more fiducial markers108. The memory has further instructions to associate scanning datacollected by the position sensitive detector 104 with the calculatedspatial position and orientation of the images from the tracking camera106 to determine a position and orientation of the scanning data usingthe known proximity of the tracking camera 106 and the positionsensitive detector 104. In some embodiments, the images and scanningdata collected can be combined to construct a (two-dimensional orthree-dimensional) model to visualize the area scanned. In the course ofuse, the hand of the user 113 grasps the handle 111 and can scan an areaof interest by moving the medical navigation apparatus 100 around anarea of interest. When the memory and processor are in an externalprocessing unit, the tracking camera 106 and the detector 104 can beconnected using one or more wires 130.

Non-limiting examples of tracking cameras useful in the medicalnavigation devices described herein include a visible light camera, anIR (infra-red) camera, a time-of-flight camera, a structured lightcamera, a stereoscopic camera, another depth sensing camera, or acombination thereof.

Non-limiting examples of position sensitive detectors are known in theart and include magnetic sensors, electromagnetic sensors, radiationdetectors. A person of skill in the art will recognize that the positionsensitive detectors can be a gamma ray probe, an ultrasound sensor, aspectroscopic camera, a hyperspectral camera, a fluorescence imager orany other known position sensitive detectors in the art.

The relative position and orientation of the tracking camera andposition sensitive detector can be adjusted based on the design needs ofthe user.

The fiducials 108 used during tracking can include markers with binarycoding applied in the proximity of the area of interest, markings addedto the body, natural marks present on a subjects body, the shape of thepatient body, or any other useful reference point of known location.There may be 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or as many fiducials asnecessary to accurately determine the location of any signals. Fiducialsthat are added to the skin may include an adhesive material that allowsfor secure fastening to the user. Added fiducials can be of any suitableshape including a square, a rectangle, an 1′ shape, a ‘V’ shape, a ‘T’shape, or a ‘U’ shape. One fiducial can be of the desired shape, ormultiple fiducials can be placed in an arrangement to form the desiredshape.

The tracking camera can be used to create a 3-D model of the patient'sbody contour by using depth imaging data, structure from motionalgorithms, or other computer vision approaches. Markings added to theskin of the patient visible to the tracking camera can improve the 3-Dmodeling performance, especially when using stereoscopic systems ormonocular systems along structure from motion type of algorithms.

In some embodiments, an object of interest is scanned instead of apatient's body.

In some embodiments, other tracking sensors, such as IMUs (inertialmeasurement units) or magnetic sensors mounted onto the body of thesensing device, or other external tracking systems, can be used toaugment the tracking capabilities of the tracking camera.

In some embodiments, the scan can be performed using a mechanical systemin which moves the apparatus with mechanical actuators that keep aprecise track of device movements. In some embodiments, these mechanicalactuators can be part of a surgical robotic system, such the Da VinciSurgical System. Other tracking modalities, such as magnetic, ultrasonicor electromagnetic trackers can be used.

FIG. 2 illustrates the use of a medical navigation apparatus 200 thatincludes a housing 202, a memory 207, a processor 209, a positionsensitive detector 204, and a tracking camera 206. The tracking camera206 has a tracking field of view 205 and is at a predetermined proximityto the position sensitive detector 204. During the scan, medicalnavigation apparatus 200 is positioned such that the position sensitivedetector 204 and the tracking camera 206 are pointed towards thesubject's body 210. The position and orientation of the tracking camera206 is determined with respect to one or more fiducials 208, andscanning data collected by the position sensitive detector 204 isassociated with the calculated spatial position and orientation of theimages from the tracking camera 206 to determine a position andorientation of the scanning data using instructions provided in thememory and executed by the processor, as described in FIG. 1. The imagesfrom the tracking camera 206 and scanning data from the positionsensitive detector 204 are combined by the processor to form a modelthat is presented on a display 214 showing a graphical user interface216 that shows the constructed model and other information collectedduring the scan. The graphical user interface 216 also include a window218 that shows a graphical representation of the depth profile 220between the signal 212 and the position sensitive detector 204 is shown.In some embodiments, the display can be any useful visualization device.

In embodiments including a graphical user interface, the graphical userinterface can include one or more windows displaying other informationcollecting during the scan. Information that can be shown on thegraphical user interface includes images from the tracking camera,scanning data displaying the relative location of any signals, and thecombined images from the tracking camera and scanning data. The combinedimages and scanning data can be represented as an overlapped image, atwo-dimensional model, or a three dimensional model.

In embodiments with more than one window in the graphical userinterface, one or more of these displays can be shown. Furtherinformation displayed in the graphical user interface windows includesgraphical plots reporting the measured signal intensity and distance aswell as a target display reporting the location of the detected signalswith respect to the alignment with the detector.

The depth profile graphical display can include a continuous plot, ahistogram plot, or any other plot that graphically represents distance.In both the continuous and histogram plot, a y-axis represents theamplitude or amount of signal detected, while an x-axis represents thedistance. In some embodiments, the axes can be reversed.

Generally, the field of view of the tracking camera is large enough tocover the area of interest from a range of positions and orientationsthat befit the application. The field of view can be altered to fit theparticular needs of the desired application.

The housing of the medical navigation apparatuses described herein canbe made out of any suitable material. That material can include metal, arigid polymer such as a thermoset plastic. Examples of such thermosetplastics include polyurethanes, polyesters, epoxy resins, phenolicresins, or copolymers of such plastics.

Although FIGS. 1-2 show the camera out of plane with the detector, theposition of the camera can be moved to anywhere to fit the needs of thedesired application. Furthermore, the orientation of the camera withrespect to its field of view can also be manipulated to fit the needs ofthe desired application.

FIG. 3 illustrates a cross sectional view of a medical navigationapparatus in accordance with an embodiment. Medical navigation apparatus300 contains an elongated housing assembly 302, gamma ray probes 304disposed at a distal end of the housing assembly, and a tracking camera306 having a tracking field of view 305 that is lateral to thelongitudinal axis of the housing assembly, the tracking camera 306having a known proximity to the position sensitive detectors 304. Thehousing assembly 302 includes a transparent optical window 307 whichprovides the tracking camera 306 a tracking field of view 305. Thehousing assembly also includes an optical fiber 322 and a transparentillumination window 324 providing an illumination field of view 326 thatoverlaps with the tracking field of view. The position sensitivedetectors 304 are within a matrix 327 that provides electrical isolationand protection from mechanical shock. In some embodiments the matrix ispresent. In some embodiments the matrix is not present. The detectors304 are operably connected to read out boards 328 that can provideelectrical power to the detectors. The boards can also host signalprocessing circuitry, such as Application Specific Integrated Circuits(ASICs) or other processors. Other arrangements of the read-out boardsand detectors are possible. For example the boards and the attacheddetectors can be placed perpendicular on the symmetry axis of thehousing. Wires 330 also connect gamma ray probes 304 and tracking camera306 to a memory 311 operatively coupled with a processor 309.

In some embodiments, this medical navigation apparatus is suitable forlaparoscopic or other intra-cavity examinations or surgeries. Thelaparoscopic device may be any useful shape. In some embodiments,intra-cavity medical navigation apparatus is cylindrical. In someembodiments, the intra-cavity medical navigation apparatus isrectangular. When used in laparoscopic surgeries or other intra-cavityprocedures, the cylindrical radius is about 35 mm, preferably 30 mm,more preferably less than 30 mm.

In some embodiments, the position sensitive detectors 304, are a singledetector. In some embodiments, there are two, three, four, or moredetectors.

The position sensitive detectors can be any useful detectors known inthe art. Non-limiting examples of detectors include a gamma ray probe,an ultrasound sensor, a spectroscopic camera, a hyperspectral camera, afluorescence imager. Examples of materials that are used for gamma rayprobes include semiconductor detectors such as silicon (Si) detectors,silicon lithium (Si(Li)) detectors, germanium (Ge) detectors, germaniumlithium (GeLi) detectors, cadmium zinc tellurium (CdZnTe) detectors,cadmium tellurium (CdTe) detectors, mercuric iodide (HgI₂), lead iodide(PbI₂), a position sensitive scintillator crystal, multiple positionsensitive scintillator crystals, segmented Si detectors, pixelatedelectrodes, parallel strip electrodes, co-planar strip electrodes,depleted CCD sensors, depleted CMOS sensors, or any other sensor knownin the art. Examples of ultrasound transducer include piezoelectriccrystal, capacitive micro-machined ultrasonic transducers (cMUTs), orany other type of ultrasonic transducer. In embodiments comprising morethan one detector, each detector is independently selected. Thesesensors are preferably of sizes around 1 cm×1 cm×1 cm, but larger orsmaller detectors can be used.

In some embodiments the transparent optical window 307 and thetransparent illumination window 324 are a single window.

In some embodiments an illumination source is provided that is not anoptical fiber. A person of skill in the art will recognize that anyillumination source can be included. In fact, the illumination sourcecan include a constant light, spatially patterned light, spectrallycoded light, time coded light, uniform light, or combinations thereof.

In some embodiments the position sensitive detector is a collimator-lessgamma ray probe with a 4 pi field of view. The memory of this radiationposition sensitive apparatus includes instructions for execution by aprocessor to convert scanning data collected by the gamma ray probe intoa reconstructed diagram identifying the location of a radiation sourcerelative to a fiducial. The reconstructed diagram can be produced fromthe scanning data using Compton imaging, self-collimation effects,proximity imaging or any known means in the art. The reconstructeddiagram can be shown in a display window in embodiments where a displayand a graphical user interface are included.

Image reconstruction of tracer distribution can be done by using Comptonimaging, self-collimation effects and/or proximity imaging. If theposition sensitive detector can provide electron track information, theshape of an least one electron track per detected event can be used toreconstruct the direction and energy of the incident radiation. Electrontracks can be used to image gamma rays as well as beta rays emitted inclose proximity to the sensor. This collimator-less camera system can beused as is, without any tracking capabilities, or tracking methods canbe used to locate the position and orientation of the imaging sensorwith respect to the body of the patient. This collimator-less imagingsensor system can be used from the outside of the body of the patient,or it can be body-insertable, such as laparoscopic. All the embodimentsdescribed above for sensor systems can apply to this collimator-lessimaging sensor system. When resolving the shape of the electron tracksis not possible, a computer operationally coupled to the sensor cancalculate a scattering angle around a scattering direction of a gammaray interacting at least two times in the sensor system by resolving thekinematics of the gamma ray interactions within the sensor system. Thekinematics is resolved by conserving the energy and momentum for Comptoninteractions taking place within the sensor system. The determinedscattering angle around a scattering direction creates a cone on thesurface of which the gamma-ray must have originated from. Byaccumulating several scan, multiple cones can be created. A statisticalimage reconstruction algorithm known in the field can be used toreconstruct the map of sources from the set of cones. For a moreaccurate image reconstruction, other factors can be accounted for, suchas attenuation in the tissue, as well as self-attenuation within thesensor. For the case when the sensor is being tracked, the computer canassociate gamma ray interaction data from the sensor with the calculatedspatial position and orientation of the sensor with respect to theadjacent object or examined object to create registered scans. The imagereconstruction will then use cones that are spatially registered in a3-D space, allowing for the reconstruction of a 3-D map of sources.

In some embodiments, the position sensitive detector and the trackingcamera are operably coupled to the memory and at least one processorwithout wires. In embodiments where no wires are used, the detector andtracking camera can be operably coupled using any known means in the artincluding a BLUETOOTH® device or a wireless router.

The transparent optical window can be made of any suitable material.Such materials include, but are not limited to glass, polycarbonate,lexan, ceramic, and other rigid clear polymers. Similarly, thetransparent illumination window can be any of these materials as well.In some embodiments, the transparent illumination window is the samematerial as the transparent optical window. In some embodiments, thetransparent illumination window is a different material then thetransparent optical window.

FIGS. 4A and 4B illustrate the use of a medical navigation apparatus inlaparoscopic surgery. The body of a patient 410 includes a small cavity432 that creates an open space to perform the surgery. The apparatus 400contains an elongated housing unit 402 that is inserted through anincision 431 into the cavity 432. The medical navigation apparatus 400includes a position sensitive detector 404 and a tracking camera 406having a tracking field of view 405 that is lateral to the longitudinalaxis of the housing assembly, the tracking camera being a knownproximity to the position sensitive detector 404. The position sensitivesensor can detect a signal 412 emanating from the scanning area.

In FIG. 4A the tracking camera 406 is located outside of thelaparoscopic cavity 432 and position and orientation information iscollected with respect to a fiducial 408 that is located outside thebody.

In FIG. 4B the tracking camera 406 is located inside of the laparoscopiccavity 432, and position and orientation information is collected withrespect to one or more fiducials 409 within the body. The medicalnavigation apparatus 402 in FIG. 4B further comprises a device 407configured to supply a stream of fluid in order to keep the transparentoptical window and the tracking field of view 405 clear.

Scanning data collected by the position sensitive detector 404 in bothFIGS. 4A and 4B is associated with the images from the tracking camera406 to determine a position and orientation of the scanning data usingthe known proximity of the tracking camera 406 and the positionsensitive detector 404. The images from the tracking camera 406 andscanning data from the position sensitive detector 404 are combined bythe processor to form an model that is presented on a display 414showing a graphical user interface 416 that shows the constructed model.

The fluid used to keep the tracking field of view of the tracking cameraclear can be any useful liquid or air. Such useful liquids include, butare not limited to, water or saline.

In embodiments including a transparent optical window, the transparentillumination window can include a device configured to supply a streamof fluid to keep the transparent illumination window clear. The fluidcan be any useful liquid or air. Such liquids include, but are notlimited to water or saline.

In some embodiments the fiducials inside the body are tissues of knownlocation. In some embodiments, the fiducials inside the body are markersplaced in known locations of the investigated area by medical personalperforming the procedure. In embodiments where fiducials are markedplaced in known locations of the investigated area, the fiducials caninclude marks with a binary coding.

Although laparoscopic surgery is an exemplary embodiment describedherein, a person of skill in the art will recognized that theapparatuses described herein can be used in various types of surgery andcan be used externally or internally. By using an internal camera,internal values such as organs can be mapped.

Although FIGS. 1-4 show the detector at the tip of the medicalnavigation apparatus, the position of the detector can vary depending onthe desired function of the apparatus. The detector can be locatedanywhere within the housing.

FIG. 5A illustrates a model showing combined images from the trackingcamera and scanning data from the detector in accordance with a scanningcompleteness embodiment of the invention. In accordance with thisembodiment, a display can show instantaneous renderings of a modelproduced from the images collected by a tracking camera and scanningdata collected by the position sensitive detector using any of thedevices described herein. The scanning sufficiency model 500 shows thebody 510 of the subject divided into gridlines 534. Also shown is afiducial 508 that has been applied to the body of the subject to performthe scan. The gridlines 534 separate the body 510 into individualvolumetric units (voxels).

A “voxel” includes an individual imaging element of volume thatrepresents separating adjacent volumetric space, or as otherwise knownin the art. Voxels can be displayed in two dimensions or threedimensions. Voxels can be broken into any useful shape includingrectangles, squares, triangles, prisms, cylinders, cones, or cubes. Insome embodiments the gridlines may be displayed on the display. In someembodiments, the gridlines are not visible on the display.

A sufficiently scanned voxel 536 appears translucent or “see through,”while an insufficiently scanned voxel 538 appears opaque. Any signals512 detected in the sufficiently scanned voxels 536 are clearly visiblein the model. As the scanning completeness increases in each voxel, theopaqueness of the voxel erodes.

FIG. 5B illustrates a cross section of a model showing combined imagesfrom the tracking camera and scanning data from the detector inaccordance with a scanning completeness embodiment of the invention. Thescanning completeness model cross section 501 of body 510 is broken upby gridlines 534 into individual volumetric units (voxels).Insufficiently scanned voxels 538 appear opaque, whereas sufficientlyscanned voxels 536 appear translucent or “see through.” Any signals 512detected in the sufficiently scanned voxels 536 are clearly visible inthe model.

Completeness of scanning in the individual voxels can be determined, forexample by assigning each voxel a scanning completeness value (SCV).Before the scanning start, the SCV may be set to zero. As each voxel isbeing scanned, its SCV value will grow indicating if the scanning ofthat particular volumetric unit is sufficient. An exemplary method ofcalculating scanning completeness value is by equating the SCV to thevoxel effective sensitivity. The voxel effective sensitivity may becalculated by summing the probability by which a signal emitted orreflected inside that volumetric unit is detected by the scanning sensorat each moment of time, over the scanning period. It is understood thatother statistical calculations of the SCV can be performed. For example,a number that represents the minimum quantity of signal that can bepresent in each volumetric unit, may be used to indicate scanningcoverage. This value may be named the Minimum Detectable Quantity (MDQ).

In some embodiments, the scanning completeness model is atwo-dimensional model of images captured by the tracking camera. In someembodiments the scanning completeness model is a three-dimensional modelof images captured by the tracking camera. The three-dimensional modelcan be shown as a 3D mesh. More extensive or complex 3-D renderings canbe also shown. In some embodiments, the scanning completeness model isan overlapped view of the scanning data and the images captured by thetracking camera.

When a particular voxel has been sufficiently scanned, the user willreceive a notification. This notification can be communicated to theuser in a variety of different means. For example, the notification canbe voxels shown on a display screen changing from opaque to “seethrough” voxels on a display screen changing color. Additionally thenotification can include the apparatus providing an audible sound to auser when a particular area has been sufficiently scanned or a lightindication on the apparatus when a particular area has been sufficientlyscanned.

Often, when a human operator performs a scanning procedure using ahand-held sensing or imaging device, coverage of the scanning area maybe incomplete or uneven, leading to incomplete or biased results.Therefore, it may be very important to provide tools to guide theoperator during the scanning process for a better quality, and morecomplete scan. For example, when using a gamma sensor such as a gammaprobe, a gamma camera, other ionizing radiation sensor or otherstatistics sensitive sensors to image tracers or markers, the magnitudeof signal taken from a certain area is proportional with the time thesensor is adjacent to that particular area, and is sensitive to thatparticular area. A larger overall signal increases the data statisticsand decreases the data noise that is fed into the image reconstructionalgorithm, having the end result of providing images with betterresolution, higher contrast and lower noise. A subject of this inventionis to describe a scanning sufficiency apparatus and method. When such astatistics sensitive sensor is moved through an environment, a trackingsystem may be used to track continuously the position and orientation ofthe sensor with respect to objects in the adjacent environment. Acomputer operationally coupled to the sensor and the tracking system cankeep track of the whole scanning history, associating scanning data withsensor position and orientation. Tracking systems used for this purposecan use mechanical displacement means, magnetic signals or waves,electromagnetic signals or waves, optical means, can use beacons or beself-sufficient. These spatially registered scans can be analyzed by thecomputer to create a 3-D distribution of sources creating the signaturesmeasured by the sensor. Iterative or analytical image reconstructionalgorithms known in the field can be used to create such 3-D maps. These3-D maps will cover a volumetric space called image space. This imagespace may be separated into small volumetric units, such as voxels, andto each voxel, a scanning completeness value (SCV) can be assigned. Thescanning tracking history may be used to calculate SCV and may informthe user about the partial volumes that have been scanned enough and thepartial volumes that have not been scanned enough for a pre-definedscanning objective. SCV can be calculated in multiple ways. In oneembodiment, the calculation of the SCV takes into account the summationof the probabilities that the signal emitted by a radioactive sourceinside the imaging element is detected by the sensor at each moment oftime over the scanning period. In another embodiment the SCV canrepresent a value that accounts for the minimum quantity of tracer ormarker that can be present in each volumetric element given the scanninghistory and the reconstructed image. In another embodiment thecalculation of the SCV takes into account the coverage of directionsfrom which the sensor observed the imaging element over the scan period.The user could be given indications about locations and orientationswhere more scans should be taken for a more complete data set. A morecomplete data set can allow formation of images that have betterresolution, better contrast, lower noise, provide detection of lowerdetectable quantities of sources, or a combination thereof. Avisualization device operationally coupled to the computer can be usedto show an instantaneous image representing a rendering of the SCV map.When a camera co-registered with the sensor is used, the visualizationdevice could show the rendering of the SCV augmented onto an essentiallylife image provided by the camera. Likewise, other renderings can becombined with the rendering of the SCV, such as a rendering of the 3-Dmap of sources, life images from cameras or other co-registered imageryor renderings.

A tag may be used to support the determination of the camera positionand orientation with respect to the body of the patient. It isunderstood that more than one tag may be used, or no tag may benecessary. In some embodiments the computer attached operationally tothe tracking camera can determine the 3-D model of the patient bodycontour. When using at least an RGB or IR camera as tracking camera,markings visible to the tracking camera can be drawn on the skin of thepatient, or stickers with markings can be placed on the skin to augmentexisting image features that may exist on the skin of the patient. Usingthe images taken by the tracking camera, computer vision specificfeature detectors, feature trackers and meshing techniques can be usedto build a 3-D model of the patient body contour. Having available the3-D mesh of the patient body, voxel occupancy techniques can beimplemented in the computer to determine what part of the space isoccupied by the body of the patient. That occupied space may beseparated in small volumetric units, such as voxels, and to each voxel,a scanning completeness value (SCV) can be assigned. However, if thecontour of the patient's body may not be determined, the whole adjacentspace may be separated in such small volumetric elements for thecalculation of the SCV. Before the scanning start, the SCV may, forexample, be set to zero. As a specific voxel from the group is beingscanned, its SCV value will grow indicating if the scanning of thatparticular pixel is sufficient. An example of calculating SCV is byequating it to the voxel effective sensitivity. The voxel effectivesensitivity may be calculated by summing the probability by which asignal emitted or reflected by the tracers or markers inside that pixelis detected by the scanning sensor at each moment of time, over thescanning period. It is understood that other statistical calculations ofthe SCV can be done. For example, a number that represents the minimumquantity of tracer or marker that can be present in each pixel, may beused to indicate scanning coverage. This value may be named the MinimumDetectable Quantity (MDQ).

In order to intuitively present to the operator the distribution ofvoxels that have been sufficiently scanned, or insufficiently, over thescanning period, a rendering of the image representing one or moreversions of SCV can be performed and displayed. In some embodiments, therendering of the SCV map can be overlapped onto the visual streamingimage taken by the tracking camera or other camera whose position andorientation is known in respect to the patient. In such embodiments, thepixel intensity from a selected area in the video stream can be modifiedto indicate skin transparency. The rendering of the SCV map may, forexample, be performed so that, voxels with incomplete scanning showtransparent, and voxels with sufficient scanning show more opaque and/orwith a higher intensity color. Various volumetric rendering or maximumintensity projection methods can be used for the visualization of theSCV map. Alternatively, the rendering of the SCV map may be performed sothat, voxels with incomplete scanning show opaque, and voxels withsufficient scanning show more transparent. An advantage of this secondvisualization approach is that the voxels that become transparent canleave a clear line of sight to visualize a rendering of the 3-D sensorcreated map that may form in the areas being scanned. The resultingvisual effect will be that, as the sensor scans a certain volume, thatvolume will become transparent (or eroded), leaving behindvolumetrically rendered colored voxels corresponding the highestintensities in the sensor produced images.

The change in opacity to full transparency may appear gradual, or indiscrete steps. Using discrete steps may help give the operator a betterimpression of volume, especially when 3-D displays are not used forvisualization. One example in which discrete steps can be used is whenonly 2 transparency values are used: full transparency and full opacity.For example, the transition between the two states may be set at a pointwhere the calculated MDQ for a specific volumetric element reaches apreset value. For example, when the volumetric element's MDQ is abovethat limit, full opacity can be associated with that volumetric element,when the MDQ is below that limit, full transparency can be associatedwith that volumetric element. The value of the preset MDQ may vary andmay be determined by the particular scope of the measurement.

In embodiments in which the 3-D modeling of the patient body contour isnot used, no pre-computed pixel occupancy may be calculated, and thewhole space around the sensing device is analyzed for scanningcompleteness, being similarly visualized.

Alternatively, or additionally to navigating the 3-D sensor producedimage by using the streaming video from the tracking camera mounted onthe sensing device, itself, the streaming video from another trackingcamera mounted on another medical instrument or tool can be used.Examples of such tools are: a surgical marking pen, a laser pointer, asurgical instrument, or a basic pen-like object.

FIG. 6 illustrates a view of a graphical user interface in accordancewith an embodiment. The graphical user interface 600 includes threewindows 642, 644, and 646, each of which present different informationto the user. Projection window 646 is showing an overlapped view ofinstantaneous images from the tracking camera and instantaneous scanningdata from the detector 604. In the projection window 646, object 640 andfiducial 608 are shown.

The contains three markings 655, 656, 657 shown in window 646 correspondto the location of signals detected by the detector 604. Graphicalwindow 642 shows a histogram plot where on the y-axis 643 the intensityof the detected signal, in this case gamma radiation, is shown and onthe x-axis 645 the distance between the source of the signal and thedetector is shown. The distance and amplitude represented by column 649corresponds to marking 655, column 648 corresponds to marking 656, andcolumn 647 corresponds to marking 657. Target window 644 shows an x, yscatter 654 of each detected signal with respect to the alignment ofeach signal with the detector. When a signal is located at the x-yintersect, the detector is completely aligned with that given signal. Inthe target window 644, detected signal 650 corresponds to marking 655,detected signal 651 corresponds to marking 657, and detected signal 652corresponds to marking 656. This view can be helpful when the operatorof the device is trying to position the detector directly in line with aparticular signal.

The graphical user interface can include any number of windowspresenting data from the scan. The windows can include images from thetracking camera, scanning data from the detector, combined images fromthe tracking camera and scanning data from the detector, a model of thecombined images and scanning data, a cross section of the model, a viewof the model from a particular angle, graphical representation of thedepth of any signals detector, or a target view showing the position ofsignals detected with respect to their alignment with the detector.

FIG. 7 shows an embodiment of the invention in which a body-insertabletelescopic optical system is used as a tracking camera by positioning itto observe the sensor system. In this example, a medical sensinginstrument 700 is body-insertable, being used in a laparoscopic surgery.The instrument 700 has a sensor 701 towards its distal end, with itspurpose being to detect specific targets within a patient 702. In thisparticular example, the surgery is laparoscopic, hence a cavity 703 isformed inside the patient by inserting gas, such as CO₂. The sensor 701can be any of the sensors described herein, or any other sensor. Anotherbody-insertable medical instrument 704 can comprise an optical systemthat can be used as a tracking camera. The field of view of suchtracking camera is represented by 705, and is oriented by the user sothat it covers the general area where the sensing device 700 operatesinside the cavity 703. The tracking camera can be used to observespecific features of the sensing device 700 housing in order to positionit in respect to the tracking camera. The tracking camera can also beused to observe the organs and tissues exposed inside the cavity 703.Consequently, the position and orientation of the sensing system 700with respect to the patient's organs and tissues can be determined. Insome embodiments a laparoscopic optical system (telescope) can be usedas medical instrument 704 housing the tracking camera system. In someembodiments, a specialized laparoscopic tracking camera system(telescope) can be used. A specialized tracking camera system cancomprise a depth imaging sensor, such as time of flight, structuredlight, monocular or stereoscopic system. A computer operationallyconnected to the tracking camera can be used to build the 3-D model ofthe organs, and to deliver the position and orientation of sensor 701with respect to the 3-D model.

In order to improve tracking performance, specific fiducial features,such as tags 706, can be positioned on the body of the sensing system700. A target of interest inside the body of the patient is representedby 707. For example, this can be cancerous tissue not yet excised, alymph node of interest, or another concentration of markers or tracers.The presence and spatial distribution of this target with respect to thepatient's organs can be determined by performing an analysis on the dataprovided by sensor 701. This analysis can be performed on a computeroperationally connected to both the sensor 701, and to the trackingcamera inside instrument 704.

The image presented to the user on a visualization device is representedby the insert 708. This image can comprise the video stream taken by theoptical system within the instrument 704. The image 709 taken by theoptical system can be fused with a rendering 710 of the image of thetarget 707 as reconstructed from the sensing data provided by sensor701. The image 711 of the sensing system 700 with a tag 706 placed on itcan also appear in the streaming video image.

While the invention has been described by way of example and in terms ofthe specific embodiments, it is to be understood that the invention isnot limited to the disclosed embodiments. To the contrary, it isintended to cover various modifications and similar arrangements aswould be apparent to those skilled in the art. Therefore, the scope ofthe appended claims should be accorded the broadest interpretation so asto encompass all such modifications and similar arrangements.

What is claimed is:
 1. An image formation and navigation apparatus,comprising: a housing assembly comprising a preferred axis physicallyidentifiable by an operator; an ionizing radiation sensor at leastpartially enclosed within the housing assembly and disposed towards adistal end of the housing assembly; an optical tracking camera at leastpartially enclosed within the housing assembly, the camera having afield of view that overlaps partially with the field of view of thesensor; a visualization device operably linked to at least one processorand configured to show instantaneous renderings of images, the at leastone processor operatively coupled with a memory, the sensor, and thecamera, the memory having instructions for execution by the at least oneprocessor configured to: determine a pose of the camera with respect toan object using an image captured by the camera and, usingtransformations derived from the camera being rigidly connected with thesensor, determine a spatial position and orientation of the sensor withrespect to the object; associate scanning data from the sensor with thedetermined spatial position and orientation of the sensor with respectto the object to create registered scans; create a 3-D map of sourcesgenerating signatures measured by the sensor by using at least tworegistered scans; and create an image for visualization on thevisualization device which is a combination of a rendered image of the3-D map and an image processed from an image captured by the camera. 2.The apparatus of claim 1, wherein the sensor is a collimated gammaimaging sensor with a divergent field of view.
 3. The apparatus of claim1, wherein the sensor is a gamma imaging sensor which providesspectroscopic and position resolution the memory having instructions forexecution by the at least one processor configured to: determine ascattering angle around a scattering direction of a gamma rayinteracting at least two times in the sensor by resolving kinematics ofthe gamma ray interactions within the sensor; and create a 3-D map ofgamma ray sources by resolving statistically an intersection of at leasttwo spatially registered cones formed by the determined scattering anglearound the scattering direction.
 4. The apparatus of claim 1, furthercomprising a tag, the at least one processor configured to determine thespatial position and orientation of the sensor with respect to theobject using an image of the tag captured by the camera.
 5. Theapparatus of claim 1, wherein the camera, or another visual camera or adepth imaging camera system at least partially enclosed within thehousing assembly, having a field of view that observes the object,provides at least one image analyzed by at least one processorconfigured to render a three-dimensional (3-D) contour of the object. 6.The apparatus of claim 1, wherein: the visualization device displays animage comprising a rendering of previously taken medical imagerycombined with an essentially live image provided by the camera.
 7. Alaparoscopic, endoscopic or intra-cavity image formation and navigationapparatus, comprising: a housing assembly having an elongated,essentially cylindrical part with a diameter of less than 30 millimeters(mm); a gamma ray sensor with spectroscopic and position resolution atleast partially enclosed within the elongated part of the housingassembly, towards a distal end of the elongated part; an opticaltracking camera at least partially enclosed within another housingassembly having an elongated, essentially cylindrical part with adiameter of less than 30 mm, towards the distal end of the elongatedpart, the camera having a field of view that observes a general areawhere the sensor is operated; a tracking and co-registration system toprovide a spatial position and orientation of the sensor and the camerawith respect to examined organs; at least one processor operativelycoupled with a memory, the sensor, the tracking and co-registrationsystem, and the camera, the memory having instructions for execution bythe at least one processor configured to: associate gamma rayinteraction data from the sensor with the provided spatial position andorientation of the sensor with respect to an object to create registeredscans; determine a scattering angle around a scattering direction of agamma ray interacting at least two times in the sensor by resolvingkinematics of the gamma ray interactions within the sensor; and create a3-D map of gamma ray sources by resolving statistically an intersectionof at least two spatially registered cones formed by the determinedscattering angle around the scattering direction.
 8. The apparatus ofclaim 7, further comprising a visualization device operably linked tothe at least one processor and configured to show instantaneousrenderings of images; wherein the memory has instructions for executionby the at least one processor configured to create an image forvisualization which is a combination of a rendered image of the 3-D mapand an image processed from an image captured by the camera.
 9. Theapparatus of claim 8, wherein the visualization device displays an imagecomprising a rendering of previously taken medical imagery combined withan essentially live image provided by the camera.
 10. The apparatus ofclaim 7, wherein the memory has instructions for execution by the atleast one processor configured to create a 3-D model of a contour oforgans by processing an image captured by the camera.
 11. The apparatusof claim 7, wherein the tracking and co-registration system providestracking and coregistration information by mechanical means.
 12. Theapparatus of claim 11, wherein the tracking and co-registration systemis part of a robotic surgery system.
 13. The apparatus of claim 7,wherein the co-registration system provides co-registration by using animage from the camera.
 14. The apparatus of claim 7, wherein the memoryhaving instructions for execution by the at least one processorconfigured to: provide tracking information for the sensor with respectto the organs by using an image from the camera, the image containing aview of a part of the sensor housing and a view of the organs, creatingthe tracking system; and provide co-registration information between theposition and orientation of the sensor and the position and orientationof the camera by using an image from the camera, the image containing aview of a part of the sensor housing and a view of the organs, creatingthe co-registration system.
 15. The apparatus of claim 7, wherein thecamera is a member selected from the group consisting of a spectroscopicimaging camera, a fluorescence imaging camera, a stereoscopic imagingcamera, a depth imaging camera, and combinations thereof.
 16. Theapparatus of claim 15, wherein a material of the sensor is a memberselected from the group consisting of a semiconductor position sensitivedetector, cadmium zinc telluride (CdZnTe) detector, a cadmium tellurium(CdTe) detector, mercuric iodide (HgI₂), lead iodide (PbI₂), a positionsensitive scintillator, a segmented silicon (Si) detector, a depletedcharge-coupled device (CCD) sensor, and a depleted complementarymetal-oxide semiconductor (CMOS) sensor.
 17. The apparatus of claim 1,wherein the memory has instructions for execution by the at least oneprocessor further configured to: create and send for visualization onthe visualization device renderings of 1-D or 2-D projections of the 3-Dmap along or perpendicular to the housing assembly preferred axis. 18.An image navigation apparatus, comprising: an elongated medicalinstrument comprising a preferred axis physically identifiable by anoperator; an optical tracking camera rigidly affixed to the medicalinstrument; a visualization device operably linked to at least oneprocessor and configured to show instantaneous renderings of images, theat least one processor operatively coupled with a memory and thetracking camera, the memory having instructions for execution by the atleast one processor configured to: load a 3D medical image dataset fromthe memory; determine a pose of the tracking camera with respect to anobject using an image captured by the tracking camera; determine thepose of the tracking camera with respect to the 3D medical image datasetusing the pose of the tracking camera with respect to the object; andcreate an image for visualization on the visualization device that is acombination of a rendered image of the 3D medical image dataset and animage processed from an image captured by the tracking camera.
 19. Theapparatus of claim 18, wherein the medical instrument comprises asurgical marking pen, a laser pointer, or a surgical instrument.
 20. Theapparatus of claim 18, wherein the 3D medical image dataset is producedby a medical imaging instrument selected from the group consisting of amagnetic resonance imaging (MM) device, a computed tomography (CT)scanner, a positron emission tomography (PET) device, a single-photonemission computed tomography (SPECT) device, a magnetic sensor, anultrasound system, a gamma camera, and a gamma probe.
 21. The apparatusof claim 18, wherein the determination of the pose of the trackingcamera with respect to the 3D medical image dataset is achieved bymatching a contour of a patient.
 22. The apparatus of claim 18, whereinthe determination of the pose of the tracking camera with respect to the3D medical image dataset is achieved by overlapping a tag.
 23. Theapparatus of claim 18, wherein the image sent to the visualizationdevice includes a virtual beam.
 24. The apparatus of claim 18, whereinthe at least one processor is configured to create and send forvisualizations on the visualization device renderings of 1D or 2Dprojections of the 3D medical image dataset along or perpendicular tothe medical instrument preferred axis.